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Foreword from CHEA 


his research describes self-reported 

perspectives of recognized institutional 
and programmatic accrediting organizations 
in the United States on student learning 
outcomes, an issue central to their future 
credibility and effectiveness. Conducted 
for the Council for Higher Education 
Accreditation (CHEA) by NORC at the 
University of Chicago in 2018, the survey 
is a continuation of CHEA’s longstanding 
commitment to support, encourage and lead 
greater capacity-building in developing and 
using evidence of student learning as it plays 
a larger role in judging academic quality. 


Several keys points emerge. Accrediting 
organizations report that they have invested 
considerable time in addressing student 
learning outcomes through accreditation 
standards, policies, accreditation visits 

and training. They have focused many of 
these efforts on accreditation processes: 
creating, applying and revising standards 
and policies, expanding formal guidance 

to address student learning outcomes and 
obtaining feedback from institutions and 
programs. Accreditors also identify student 
learning outcomes as playing a significant 
role in their formal actions (e.g., award or 
deny accreditation), although this role is not 
always determinative. Accreditors also believe 
that the majority of their programs and 
institutions are providing adequate evidence 
of student achievement and that the quality 
of this evidence has improved. Accreditors 
indicate some concern that the external 
review of the quality and effectiveness of 
their organizations, through CHEA or the U.S. 
Department of Education (USDE), may move 
them away from their preferred approaches 
to assessing student learning outcomes. 


The national conversation about accreditation 
during the past several years includes 

strong signals that accreditors need to take 
significant additional steps to make evidence 
of student learning central to judgments 
about academic quality and accredited 
status. There is a growing body of evidence 
that a considerable percentage of the 

public, government and media may not find 


accredited status to be a credible indicator 
that institutions and programs are serving 
and protecting students well. Additionally, 
leaders in both political parties, in an effort to 
safeguard students and the federal student 
aid investment, have asserted a need for 
greater federal regulation and oversight. 


To continue to respond to these challenges, 
accreditors can take additional steps beyond 
the impressive work described here to 
provide additional evidence and transparency 
about student learning outcomes: what 
students learn, student success with regard 
to achieving stated educational goals, 
completion of degrees and other credentials, 
successful transfer of credit, successful 

job acquisition, longer-term advancement 
and earnings. The future credibility and 
effectiveness of accreditation is likely 
dependent on such action, preferably led by 
the accreditation and academic communities: 


Evidence of Student Learning 

Further emphasize and make public what 
accreditors consider as robust and reliable 
evidence of student learning outcomes. 


Acceptable Levels of Student Success 
Identify and make public explicit levels of 
success with regard to student learning 
outcomes for institutions and programs, 
contingent on mission and student 
population, as a central requirement to obtain 
and maintain accredited status. 


Expeditious Action to Counter Substandard 
Performance 

Develop and apply means to identify and act 
expeditiously on institutions and programs 
with substandard performance in relation 

to student achievement, as defined by the 
accreditor and its institutions and programs. 


Timely Action to Improve Performance 
For institutions and programs that need 

to improve their performance but are not 
considered substandard with regard to 
student learning outcomes, assure that 
improvement takes place in a timely way in 
order that students are not harmed. 
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Introduction 


This report presents findings from a 

2018 survey, CHEA Survey of Accrediting 
Organizations, administered by NORC 

at the University of Chicago (NORC), an 
independent research organization, on 

behalf of the Council for Higher Education 
Accreditation (CHEA). The purpose 

was to gather insights from accrediting 
organizations about their recent experiences 
in two areas: 1) student learning outcomes 
and 2) innovation in accreditation. The 
findings from the survey are being distributed 
via this research report focused on accreditor 
practices associated with student learning 
outcomes and via a companion report on 
innovation in accreditation. 


This paper examines accreditor perspectives 
on four broad topics related to student 
learning outcomes: 


1. How are accreditors currently 
addressing student learning outcomes? 
2. What has been the impact of 
accreditation standards and policies on 
the evidence of student achievement? 
3. Is the evidence of student 
achievement affecting the accreditation 
status of institutions and programs? 

4. What is working, needs improvement 
and provides concern among 
accreditors about student achievement? 


Operational Definitions Related to 
the Research 


For the purpose of this research, the survey 
instructed respondents to consider the 
terms “student learning outcomes” and 
“student achievement,” used interchangeably 
throughout, to refer to the knowledge, skills 
and abilities that a student has attained as 

a result of engagement in a particular set of 
higher education experiences. 


U.S. recognized institutional 

and programmatic accrediting 
organizations report that they have 
made a considerable investment and 
have produced impressive results in 
addressing student learning outcomes 
in higher education institutions and 
programs. 


Accreditors tell us that the primary 
focus of their attention to student 
learning outcomes has been centered 
on their processes, standards, training 
and communication with their member 
institutions and programs. 


Accreditors indicate a strong interest 

in further work on student learning 
outcomes —seeking to continue to 
address this vital issue to serve students 
and society. 


CHEA is focused on further assisting all 
recognized accrediting organizations 
through efforts to establish additional 
evidence to the public of what 

counts as student learning, to further 
articulate acceptable levels of student 
success to achieve accredited status, 
to further examine and act to counter 
substandard performance if found in 
accredited institutions and programs 
and to take even more timely action to 
improve performance in all accredited 
institutions and programs. 
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Background on 
Accreditation and 
Student Learning 
Outcomes 


Attention to student achievement in higher 
education accreditation has intensified in 
recent years, but is not a new phenomenon. 
In 1989, the USDE began requiring accrediting 
organizations to address student learning 
outcomes as a condition for recognition, 
and by 1989, half of all states required public 
institutions to assess and publicly report on 
student learning. By the mid-1990s, most 
accrediting organizations were examining 
student achievement in the institutions and 
programs they authorized, though their 
approaches, levels of engagement and 
terminology varied widely.' 


CHEA, which was formed in 1996, began 
focusing on the role of accreditation in 
addressing student achievement with the 
publication of the 2001 report, Accreditation 
and Student Learning Outcomes: A Proposed 
Point of Departure. In 2006, CHEA introduced 
its Award for Outstanding Institutional 
Progress in Student Learning Outcomes,” 
which annually recognized outstanding 
practices in developing and applying 
evidence of student learning outcomes 

as part of the ongoing accountability and 
improvement of college and university 
programs of study. CHEA continues its 

focus on student learning outcomes 

through research and policy analysis, 
convening meetings and other national and 
international efforts to identify solutions and 
build consensus around advancing student 
achievement. 


In commissioning this research, CHEA 
wanted to reach out directly to recognized 
accreditors in both the programmatic and 
institutional domains to gather insights 

from those who have been involved in the 
ever-evolving efforts to demonstrate and 
document evidence of student achievement 
in accreditation. The candid and unattributed 
survey responses provide insights into how 


student achievement is currently fitting into 
the quality assurance system across U.S. 
higher education. 


Research Methods 


NORC at the University of Chicago invited 
the combined pool of 86 CHEA- and USDE- 
recognized accrediting organizations to 
participate in a self-administered web survey 
between July 10 and August 13, 2018. 


¢ Sixty-four out of 86 accreditors 
responded to the survey for an overall 
response rate of 74 percent. 


¢ Thirteen out of 19 institutional 
accreditors responded (68 percent). 
Institutional accreditors included five 
regional, five national career-related and 
three national faith-related accrediting 
organizations. 


¢ Fifty-one out of 67 programmatic or 
specialized accreditors responded (76 
percent). 


¢ Data on accrediting organizations 
published in the CHEA Almanac 

Online? indicate that accreditors who 
responded to the survey represent more 
than 6,000 institutions and more than 
22,000 specialized programs. 


* To ensure participant confidentiality, 
NORC was responsible for sending 

all survey invitations and follow-up 
emails. Communications emphasized 
NORC’s nonbiased role in collecting and 
analyzing the data on behalf of CHEA. 
The link between the survey URL and 
the participant list was maintained on 
NORC’s secure servers and was not 
disclosed to CHEA staff. 
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Findings 


Part 1: How Are Accreditors 
Currently Addressing 
Student Learning 
Outcomes? 


The survey probed accreditors on their 
current standards and policies that address 
student learning outcomes and achievement. 
The responses offered important insights 
about accreditor approaches, encompassing 
standards, official policies and other guidance 
as well as changes undertaken in recent 
years. 


Accreditation Approaches Are 
Informed by Many Factors 


Institutional and programmatic accreditors 
cited a number of factors as important to 
informing their approaches to examining 
student learning and achievement (Exhibit 1). 
Every institutional accreditor reported that 
feedback from the institutions they serve is 

a very or extremely important factor in their 
approach to examining student achievement; 
92 percent indicated that recognition 
requirements, federal policy and students 
and their families are also very or extremely 


important factors. Programmatic accreditors 
cited feedback from member programs 

(85 percent), recognition requirements (79 
percent) and feedback from employers (63 
percent) most frequently as being extremely 
or very important factors for informing their 
organization’s approach. 


Emphasis on Student Achievement 
Varies in Accreditation Standards, 
Policies and Guidance 


Eighty-two percent of institutional 
accreditors reported that accreditation 
standards explicitly addressing student 
learning outcomes or student achievement 
account for roughly a quarter or less of 

their accreditation standards. In contrast, 
standards addressing student achievement 
generally make up a larger share of 
programmatic accreditation requirements, 
with 43 percent of programmatic accreditors 
indicating they represent a quarter or less 
and 57 percent indicating that they represent 
more than a quarter of all accreditation 
standards (Exhibit 2). Each accreditor 
determined the proportion of its standards 
that address student achievement. 


In addition to accreditation standards, a 
substantial majority of accreditors - 83 
percent of institutional accreditors and 
71 percent of programmatic accreditors - 
also hold their institutions and programs 


Exhibit 1. Rate the following in terms of importance for informing your organization’s 
approach to student learning outcomes or student achievement and accreditation 


Institutional Accreditation 


Programmatic Accreditation 


Feedback from members 
CHEA or USDE recognition requirements 
Other accrediting organizations 
C-RAC principles 
ASPA guidance 
Research 
Federal policy 
State policies 
Higher education associations 
Students and their families 
Feedback from employers 


Very or extremely important 


Moderately or slightly important Moderately or slightly important 
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accountable for policies or other guidance Exhibit 3. Has your organization made significant 


related to student learning outcomes or changes to standards, policies, other formal 
student achievement. guidance or evaluation practices related 

to student learning outcomes or student 
Exhibit 2. Roughly what share of your achievement in the last five years? 


organization’s accreditation standards explicitly 


address student learning outcomes or student 75% 
achievement? Significant changes to 78% 
accreditation standards 7 

62% 
43% aaa ast 59% 

0-25% Significant changes to policies 57% 

82% or other formal guidance = 
40% 
26-50% Significant changes to 
9% evaluation practices 


9% % 

51-75% ° No significant changes pee 
9% 

8% 


More than 75% ie 8% 


69% 


33% 
32% 
39% 


= Overall @& Programmatic ® Institutional 


mIPro gemma: inst tuianal Programmatic accreditors also made 
significant policy changes to clarify intent (79 
Accreditors Continue to Refine percent), foster transparency (69 percent) 


Approaches to Finding Evidence of and improve the rigor of their programs’ 
Student Achievement approaches (22 percent) 


Both institutional and programmatic 
accreditors made significant changes to their 
evaluation practices to improve the rigor 
with which institutions and programs provide 
evidence of student achievement, as well as 
to simplify, consolidate or better organize 
their student achievement approaches 
(Exhibit 4). 


Most accrediting organizations reported 
making significant changes in their 
approaches to examining student 
achievement in the last five years. Seventy- 
five percent of accreditors made significant 
changes to standards, 59 percent changed 
policies or other official guidance and 33 
percent changed their evaluation practices 
(Exhibit 3). Each accreditor determined 
what counts as a “significant change” in this 
context. 


Most institutional accreditors made significant 
changes to standards in order to clarify their 
intent (88 percent) or to respond to feedback 
from institutions (63 percent). Institutional 
accreditors also changed their policies and 
guidance on student achievement mainly to 
clarify their intent (89 percent), to improve 
the rigor of their institution’s approaches (78 
percent), to simplify or consolidate policies 
(67 percent) and to respond to USDE or 
CHEA recognition requirements (67 percent). 
Programmatic accreditors reported making 
significant changes to standards largely to 
clarify intent (75 percent), improve rigor (65 
percent), simplify (65 percent) and foster 
transparency (60 percent). 
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Exhibit 4. For what reasons did your organization make significant recent changes to accreditation 
standards, policies, other formal guidance or evaluation practices related to student learning outcomes 
or student achievement in the last five years? (Applies only to accreditors that reported making 


significant changes, as depicted in Exhibit 3) 


Changes to Accreditation 


Changes to Policies and Changes to Policies and 


Standards Formal Guidance Evaluation Practices 
Clarify the intent of standard(s) 
Improve rigor of approaches to meeting standard(s) 
Foster transparency (e.g., share student outcomes info) 
Simplify, consolidate or better organize the standard(s) (34/4 50% 45% 67% 75% 80% 

Respond to related changes in federal policy 28% 33% 20% 

Respond to changes in higher ed (e.g., demographics, tech) 
Respond to USDE or CHEA recognition requirements 48% 50% 
Respond to feedback from members 50% 63% ae Ee 

Respond to feedback from public 10% so% 19% 

Other reason(s) 5% I 13% 3%] 6%] 


Part 2: What Has Been the 
Impact of Accreditation 
Standards and Policies on 
the Evidence of Student 
Achievement? 


The survey asked accreditors about the 
extent to which institutions and programs 

are providing evidence of student learning 

as required in the accreditation process. 
Questions addressed common sources of 
evidence of student achievement and sources 
that are encouraged or discouraged by the 
accreditor, perceived quality of the evidence, 


= Programmatic & Institutional 


whether accreditors prescribe thresholds 
established either by the accreditor or 
institution/program for any of their indicators 
of student achievement and public availability 
of evidence of student learning outcomes 
and achievement provided by institutions and 
programs. 


Institutions Use Multiple Sources of 
Evidence of Student Learning 


The survey asked accreditors to rank the 
sources most often used by their institutions 
or programs as evidence of student learning 
and achievement. Exhibit 5 shows a summary 
of top-ranked sources: Programmatic 


Exhibit 5. What sources do your institutions or programs most often cite as evidence of student 
learning outcomes and achievement? (Chart shows the abso/ute number of accreditors that ranked each 


source Ist or 2nd from a list of choices) 


Programmatic Institutional 


National student surveys (e.g., NSSE, CIRP, CCSSE, etc.) | 3 5 
Standardized exams, including certification and licensure exams 28 2 
(e.g., ETS Major Field Tests, Collegiate Learning Assessment+, Praxis, etc.) 

Internally developed student surveys | 10 2 
Internally developed exams 10 Oo 

Graduation or retention rates | 19 7 
Job and graduate school placement rates 16 4 

Student portfolios 9 2 

Course grades 4 1 


COUNCIL FOR HIGHER EDUCATION ACCREDITATION/CHEA INTERNATIONAL QUALITY GROUP | 9 


accreditors largely ranked standardized 
exams, graduation or retention rates and 
job/further education placement rates 
among the top sources of evidence, while 
institutional accreditors ranked graduation 
rates, national student surveys (e.g., National 
Survey of Student Engagement) and job/ 
further education placement rates as the 
sources of evidence most often cited by their 
institutions. 


Most Accreditors Encourage Specific 
Indicators as Evidence of Student 
Learning Outcomes and Achievement 


In addition, 92 percent of institutional and 
89 percent of programmatic accreditors 
responding to the survey indicated they 
require or encourage specific indicators as 
evidence of student learning or achievement. 
Exhibit 6 shows that the most common 
indicators they encourage or require 

from their institutions and programs are 
graduation/completion rates, certification/ 
licensure or other standardized exam passing 
rates and job placement rates. 


Exhibit 6. Please list the indicators that your 
institutions or programs are required or 
encouraged to use as evidence of student 
learning or achievement. 


Graduation or completion rate 


Retention rate 36% 


Cert/licensure/other exam pass rate 9 


Subsequent education rate 


% 
Job placement rate 62% 42 


Loan default rate 8% 


Transfer out rate 


D 
5% | 17% 
» 
2% | 8% 
” 


@ Programmatic @ Institutional 


Stakeholder feedback 


Portfolio 


76' 83% 
50% 
74 50% 
% 
9 
% 
% 
% 


Other 


Furthermore, 15 percent of institutional and 
23 percent of programmatic accreditors 
reported that they explicitly discourage or 
prohibit particular sources of evidence as 
part of the accreditation review process. 
Course grades and indirect measures like 


student satisfaction were the most commonly 
cited sources of discouraged or prohibited 
evidence. 


Nearly Half of Accreditors Use 
Performance Thresholds 


The survey asked accrediting organizations 
whether they require institutions and 
programs to meet bright-line student learning 
outcomes in the context of quantitative 
performance thresholds for particular 
indicators related to accreditation. Most 
programmatic accreditors require programs 
to meet thresholds either set by the 
accreditors (54 percent) or set by programs 
themselves (23 percent). These thresholds 
are often associated with indicators such 

as certification or licensure exam passing 
rates and graduation or completion rates. 
This finding is consistent with a 2016 report? 
by the Association of Specialized and 
Professional Accreditors (ASPA) that found 
52 percent of ASPA members had bright-line 
outcomes requirements. 


In comparison, 46 percent of institutional 
accreditors reported setting bright line 
performance requirements, while eight 
percent require institutions to set their 

own threshold requirements. This likely is 

a reflection of the USDE requirement for 
for-profit institutions, accredited by career- 
related national accreditors, to meet gainful 
employment standards based on the student 
debt-to-earnings ratio. 


Accreditors Report Majority of 
Institutions Provide Adequate 
Evidence of Student Learning 
and that Quality of Evidence Has 
Improved 


Accreditors were asked to estimate the 
percentage of their institutions and programs 
that provide “adequate evidence of student 
learning outcomes and achievement” in the 
accreditation review process, consistent 

with accreditor standards, policies and 
guidance. According to the accrediting 
organizations surveyed, the vast majority of 
institutions and programs currently provide 
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Exhibit 7. Roughly what share of your members provide adequate evidence of student learning 
outcomes and achievement in the accreditation review process? 


Approximate percentage of members cere abot nceponeing 


providing adequate evidence 


Programmatic Institutional 


adequate evidence of student learning 
outcomes (Exhibit 7). Nearly three-quarters 
of programmatic accreditors and a little more 
than half of institutional accreditors indicated 
76 percent to 100 percent of their programs 
and institutions provide adequate evidence. 
However, a quarter of programmatic 
accreditors and nearly half of institutional 
accreditors reported that just 51 percent to 
75 percent of their programs and institutions 
provide adequate evidence of student 
learning, indicating there is room for further 
development. 


Accreditors were also asked their perception 
about whether the overall quality of student 
outcomes evidence was better or worse than 
compared to five years ago. Exhibit 8 shows 
that 90 percent of programmatic accreditors 
and all of institutional accreditors reported 
that the quality of evidence has gotten 

much or somewhat better over the past five 
years, and none reported that the quality of 
evidence has become worse. Each accreditor 
determined what counts as “overall quality.” 


Exhibit 8. To what extent has the overall quality 
of evidence of learning and achievement that 
your institutions or programs provide through the 
accreditation process changed over the past five 
years? 


Institutional 


Much better Mi Somewhat better M About the same 


Programmatic 45% 45% 10% Institutional 31% 
54% 46% 


Institutions, Programs and 
Accreditors Are Making Evidence 
of Student Learning Outcomes 
Available to the Public 


Evidence of student learning and 
achievement that institutions and programs 
provide for accreditation is often shared with 
the public through websites, press releases 
and other publications. Nearly 80 percent 

of programmatic accreditors and two-thirds 
of institutional accreditors reported that 76 
percent to 100 percent of their programs 

or institutions publish student learning and 
achievement outcomes (Exhibit 10). Moreover, 
the majority (55 percent) of programmatic 
and nearly a third of institutional accrediting 
organizations themselves publish evidence 
of student achievement provided by their 
programs and institutions (Exhibit 9). 


Exhibit 9. Does your accrediting organization 
make evidence of student learning or 
achievement from your member institutions or 
programs publicly available (e.g., on your website, 
in publications or in press releases)? 


Programmatic 


@ Yes HM No 
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Exhibit 10. Roughly what share of your institutions or programs make evidence of student learning 
or achievement publicly available (e.g., on the institution’s or program’s website, in publications or in 


press releases)? 


Approximate percentage of members 
publishing evidence 


Accreditors Offer Support to 

Institutions and Programs Through 
Training and Resources Related to 
Evidence of Student Achievement 


Most of the accrediting organizations that 
responded to the survey offer resources 

to assist their members in addressing 
accreditation requirements related to student 
learning outcomes and achievement. While 
29 percent of programmatic accreditors 
indicated that they do not offer formal 
training, 100 percent of institutional 
accreditors and the remaining 71 percent 

of programmatic accreditors offer training 
specifically related to student learning 
outcomes and achievement at least once per 
year, with some offering training several times 
per year. In fact, accreditors offer a selection 
of resources related to student achievement, 
as illustrated in Exhibit 11. Institutional 
accreditors primarily offer in-person 
workshops and special sessions both during 
(92 percent) and outside of (77 percent) 
annual meetings, as well as supplementary 
informational materials (77 percent) and 


Programmatic 


76 - 100% 


VN olol ¢-Ye lide) aM a-sJelelilel lite) 


Institutional 


informal guidance as needed (77 percent). 
Programmatic accreditors primarily offer 
informal guidance as needed (85 percent), 
supplementary materials (69 percent) and 
workshops at annual meetings (65 percent). 


Part 3: Is the Evidence 

of Student Achievement 
Affecting the Accreditation 
Status of Institutions and 
Programs? 


The survey examined what happens when 
accreditors determine that evidence of 
student achievement provided by institutions 
and programs does not satisfy requirements, 
including accreditor perspectives about the 
extent to which student learning outcomes 
are considered in accreditation actions. 


As part of their reviews of institutions and 
programs for academic quality, accrediting 
organizations take formal actions to 

grant, reaffirm, defer, deny or withdraw 


Exhibit 11. Which of the following resources do you offer member institutions or programs to assist 
them in addressing accreditation requirements related to student learning outcomes and achievement? 


Informal guidance and support on an as-needed basis 
Web resources such as bibliographies 


Networking opportunities 


7 7, 
SS SSS 


51% 
——SSSSSSSSerreg 


84% 
41% 


es 54%, 


es 41%, 


Supplementary materials that provide guidance 


es 7 7%, 


Es 69% 


Assessment experts on accreditation review teams 62% 
Ds 29% 
Specialized training or workshop opportunities SS 77% 
_——L—SEEe fo 


In-person workshops at annual meetings 


92% 


ee 65°, 


Other resources 


me 8% 


mm 4% 


Institutional & Programmatic 
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accreditation. They also issue show cause or 
probation orders. Accrediting organizations’ 
definitions of these terms may vary; 
nevertheless, CHEA annually compiles data 
on these actions from information published 
on individual accreditor websites in the CHEA 
Almanac Online.® Exhibit 12 provides CHEA’s 
definitions of formal accreditation actions. 
Exhibit 15 presents 2017 A/manac Online data, 
applying CHEA’s definitions of formal actions 


to information obtained from the websites of 
accrediting organizations. 

These data illustrate that a substantial 
majority of formal accreditation actions 
reaffirm or grant accreditation status, 

with relatively few sanctions issued in 

a given year. For example, 1,059 out of 

1,751 (60 percent) institutional and 3,098 

out of 3,627 (84 percent) programmatic 
accreditation actions reaffirmed or granted 


Exhibit 12. CHEA definitions of formal actions taken by accrediting organizations based on whether 
institutions and programs demonstrate that accreditation requirements are met. 


A decision to award an initial accreditation status when 
an institution or program demonstrates that accreditation 
requirements are met. 


Grant Accreditation 


A decision to postpone action on a program or institution 
until receipt of additional information that shows accreditation 
requirements are met as specified by the accreditor. 


; as A decision to continue accreditation when an institution or 
Reaffirm Accreditation | program demonstrates that accreditation requirements are met. 


A determination that although one or more accreditation 

requirements have not been met, the institution has the capacity 
to make appropriate improvements within a reasonable period of 
time and fully meet the accreditation standards for the long term. 


Defer Accreditation 


Notice/Warning 


A determination that accreditation requirements have not been 
met and the accreditor mandates corrective action to resolve one 
or more deficiencies within a specified period of time. Show Cause 
actions may be made public by the accreditor. 


Show Cause 


A determination that accreditation requirements have not been 
met and the accreditor mandates corrective action to resolve one 
or more deficiencies within a specified period of time. Probation 
actions may be made public by the accreditor. 


Probation 


A determination that an institution or program does not 
demonstrate it can meet accreditation requirements and is denied 
a grant of initial accreditation. Accreditors permit the appeal of 
this decision. 


Deny Accreditation 


A determination that an institution or program is unable to 
demonstrate it can meet specified accreditation requirements and 
is removed from accredited status by the accreditor. Accreditors 
permit the appeal of this decision. 


Withdraw 
Accreditation 
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Exhibit 13. Summary of formal accreditation actions in 2017, by organization type. 


Grant accreditation 10 16 117 1,079 1,222 
Reaffirm accreditation 339 38 539 2,019 2,935 
Deny accreditation fe] 27 4 31 
Withdraw accreditation 187 144 333 
Defer accreditation 1 88 204 293 
Notice/warning 34 1 160 59 254 
Show cause 4 2 99 4 109 
Probation 22 3 62 114 201 
Appeals Oo (e) (0) Oo ie) 
Totals 409 63 1,279 3,627 5,378 


NOTES: Data compiled from individual accreditor websites. The terms used to describe formal 
actions are based on CHEA’s definitions (Exhibit 12). 


Source: CHEA Almanac Online 


initial accreditation status in 2017 (Exhibit 
13). Critics of accreditation have interpreted 
statistics like these (i.e., low sanction rates) 
to suggest that accreditors are “watchdogs 
that rarely bite”’® or that they “need higher 
standards”’” for overseeing low-performing 
institutions and programs. A 2015 report of 
the U.S. Government Accountability Office 
(GAO) found that accreditors most frequently 
issued sanctions for failure to meet standards 
related to financial capability rather than 
student achievement and that schools with 
stronger student outcomes were as likely 

to be sanctioned by accreditors as schools 
with weaker student outcomes.® In 2016, the 
National Advisory Committee on Institutional 
Quality and Integrity, which advises USDE 

on accreditation issues, piloted “accreditor 
dashboards” for more systematic, outcomes- 
focused reviews of accreditors based on their 
institutions’ student outcomes data, such as 
graduation rates, earnings and loan default 
rates.? 


Accreditors responded with efforts like the 
Council of Regional Accrediting Commission’s 
Graduation Rate Information Project, which 
explored strategies for identifying and 


providing effective oversight of academically 
struggling institutions."° 


As attention to student outcomes and 
accreditation actions has increased in 
recent years, most of the focus has been on 
the formal actions in Exhibit 13. However, 
accrediting organizations also provide 
oversight of institutions and programs 

that fall short of student achievement 
requirements in ways that do not involve 
formal sanctions. Other accreditor actions 
might include requiring monitoring reports 
on student learning outcomes, interim reports 
or site visits addressing certain standards 

or reaffirming accreditation for a shorter 
amount of time. And unlike the accreditation 
actions listed in Exhibit 13, these actions 

are not considered sanctions and are not 
typically disclosed to the public. It is difficult 
to determine the scope and frequency of 
such actions or to compare them across 
accrediting organizations. 
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Programmatic 


Institutional 


Accreditation Actions Consider 
Failure to Provide Adequate 
Evidence of Student Learning 
Outcomes and Achievement 


Nineteen percent of programmatic and 23 
percent of institutional accreditors reported 
that more than half of their informal and 
formal accreditation actions in 2017 were 

in part due to inadequate evidence of 
student achievement. Thirty-eight percent of 
programmatic and 54 percent of institutional 
accreditors indicated that a quarter or fewer 
of their informal and formal accreditation 
actions were partly due to problems 

with demonstrating evidence of student 
achievement, suggesting that the plurality 

of actions are still based at least in part on 
factors other than student achievement. 
Focusing specifically on removal or denial 

of accreditation status over a longer period 
of time, 40 percent of programmatic and 54 
percent of institutional accreditors responded 
that they had taken such an action in the last 
five years due in part to problems associated 
with evidence of student achievement 
(Exhibit 14). 


Exhibit 14. In the past five years, have any of 
your institutions or programs lost or been 
denied accreditation due, at least partially, to 
problems with evidence of student learning or 
achievement? 


BYes NO 


Part 4: What Is Working, 
Needs Improvement and 
Causes Concern Among 
Accreditors About Student 
Achievement? 


The survey asked accreditors to reflect on 
student outcomes and accreditation along 
three specific dimensions - what is working, 
what needs improvement and what most 
concerns accreditors about the future. 


What Is Working? 


In describing what they felt was working 
well, both institutional and programmatic 
accreditors highlighted training, coaching 
and workshops they provide as well as the 
detailed guidance and clear expectations 
they provide. Institutional accreditors 
repeatedly noted that their review processes 
and evaluation strategies are also working 
well, while programmatic accreditors 
emphasized that flexibility in defining 
student outcomes and publishing student 
achievement metrics on accreditor and/ 

or program websites are working well. 
Exhibit 15 shares illustrative comments from 
respondents. 


What Needs Improvement? 


In describing what needs improvement with 
respect to how accreditors incorporate 
student achievement into the accreditation 
process, both institutional and programmatic 
accreditors frequently mentioned the need 

to improve the quality of measures used to 
demonstrate evidence of student learning 

and achievement. Respondents underscored 
the challenges of institutional and program 
diversity and recognition requirements 

from CHEA and the USDE. Some pointed 

to perceived tensions between the use of 
qualitative “student learning” and quantitative 
“student achievement.” Institutional 
accreditors also noted the need for clearer 
expectations, more detailed guidance, 
strategies for sharing best practices and more 
training, both internally and for constituent 
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Exhibit 15: What is Working, What Needs Improvement, What Concerns Accreditors About the Future: 
Illustrative comments from Institutional and Programmatic Accreditors 


Institutional Accreditors 


Programmatic Accreditors 


What is working? 


Increased attention to Student Learning 
Outcomes (SLOs), including at the program, 
degree and institutional levels. Training of 
institutional assessment leaders to expand and 
integrate effective use of outcome data into 
review and improvement across campuses. 


Institution-Set Standards (ISS) are an aspect of 
each comprehensive review. Teams evaluate the 
process used to set these standards and the data 
sets by which their achievement is determined. 
This has prompted many beneficial conversations 
that have kept a focus on student achievement. 


Validating student achievement metrics and 
posting them on our website and then requiring 
programs to have a link on their website to the 
outcomes we publish. 


Standardizing the request of data so upon receipt 
of the program’s information, we can compare 
apples to apples—which ultimately will provide us 
with a baseline for setting specific benchmarks 
that the commission deems reasonable and 
relevant. 


What needs improvement? 


We found it essential to distinguish between 
SLOs—based on qualitative measures, assigned 
by faculty, unique to programs and institutions, 
thus not comparable among members; and 
Student Achievement (graduation, retention, 
transfer, persistence)—thus quantitative, captured 
by the Institutional Research (IR) Department. 
This was essential in terms of expectations for 
using disaggregated data to watch for gaps in 
demographic student populations. 


Our current strategic plan has a large component 
on Student Success and we are updating our 
criteria. We have a big national think tank of 
higher education innovators also looking at these 
issues. 


Would be extremely helpful if CHEA and ASPA 
specified the data they feel most impactful so 
that...we can all compare apples to apples. Now, 
every accreditor “assumes” what they believe to 
be relevant. However, what we think is relevant, 
may not be to someone else. 


Trying to balance the requirements for tracking 
and reporting numerical outcomes (such 

as certification exam pass rates) with more 
descriptive outcomes (such as impact of the 


program and graduates on the community) is 
difficult. Our process for reviewing programs that 
do not meet benchmarks is new and still a little 
clunky. 


What concerns you the most about the future? 


Greatest concern is that student achievement will 
be established without regard for the student 
population served and the level of collegiate 
preparation held when enrolling. Also, that the 
drive to raise graduation rates will actually exclude 
First-generation college students and those socio- 
economic groups who are under prepared and 
under-served by higher education. 


That the focus will be primarily on outcomes 
that do not relate directly to student learning. 
For example, federal policy has pushed hard in 
the last few years almost exclusively on retention 
rates, graduation rates, student debt amounts, 
loan default rates and placement in positions 
with significant salaries. None of those measures 
directly address student learning, and most are 
focused more on economic or financial benefits, 
which is not a priority for the theological schools 
that our agency accredits. 


Standardization. That programs will end up 
coming out of “cookie cutters” to meet some 
standardized performance that does not factor 
in the unique character and needs of the 
communities. That this will quench what has 
been our profession’s strongest suit—education 
innovation. 


Forcing accreditors to release more information 
or forcing accreditors to require institutions to 
release more information will not serve students 
better or help protect the public. The USDE or 
CHEA becoming more prescriptive on this matter 
is a significant concern. Already there has been 
significant variability and inconsistency in the 
ways in which the USDE and CHEA have reviewed 
and cited...accrediting agencies on matters such 
as student achievement and transparency. Not 
once has a prospective student called our offices 
because they weren’t sure whether to enroll in 

an institution because of the completion rate, the 
employment rate or the pass rates. 
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institutions. Programmatic accreditors also 
cited training and information sharing needs 


as well as more consistency and better review 


processes. Many programmatic accreditors 
mentioned that they are currently monitoring 
or evaluating their approaches to student 
outcomes and accreditation to identify 
potential areas for improvement. 


What Concerns Accreditors Most? 


In expressing what concerns them most 
about the future of student learning 
outcomes and accreditation, institutional 

and programmatic accreditors were nearly 
unanimous in their concern about the 

trend toward standardized “cookie cutter” 
definitions and approaches as well as the use 
of “blunt” measures without regard for the 
rich heterogeneity of the missions, cultures 
and student populations of institutions and 
programs. Several institutional accreditors 
expressed concerns that such a shift will 
reduce access and stifle innovation in higher 
education. Programmatic accreditors also 
raised concerns that overemphasis on 

rigid or prescriptive student achievement 
requirements, which may not actually reflect 
learning, will ultimately diminish attention 

to other meaningful aspects of students’ 
education experiences. Both accreditor types 
mentioned concerns about external pressures 
from USDE and CHEA, data collection 

and management and keeping pace with 
rapid changes in higher education, such as 
education delivery modalities. 


Conclusions and 
Implications 


The 64 recognized accrediting organizations 
that participated in this survey offered 
important perspectives about how they are 
addressing student learning outcomes. The 
self-reported insights suggest that there 

has been significant evolution in the role 
that accreditors play in encouraging the use 
and improvement of evidence of student 
achievement and learning outcomes through 
the accreditation process in recent years. 


Key takeaways on how accreditors reported 
they are currently addressing student 
learning outcomes included: 


* Accrediting organizations address 
student achievement through standards, 
policies and formal guidance. 

* Programmatic accreditors reported that 
student achievement-related standards 
represent a larger percentage of their 
overall standards, on average, than 
institutional accreditors reported. 

e« Institutional accreditors’ approaches 
to student achievement are largely 
informed by feedback from institutions, 
recognition requirements, federal 
policy and students and families. 
Programmatic accreditors largely 
consider feedback from programs, 
recognition requirements and feedback 
from employers. 

« Nearly all accreditors reported making 
significant changes in recent years to 
standards, policies and guidance and 
evaluation practices related to student 
achievement. Many of these changes 
were made to clarify intent, improve 
the rigor of institution and program 
approaches, foster transparency, 
simplify requirements and respond 
to feedback from institutions and 
programs. 


A majority of the respondents reported a 
two-way communication with institutions 
and programs to incorporate and refine 

the sources of evidence of student 
achievement and how they are integrated 
into the accreditation process. Additionally, 
accreditors expressed that the voluntary, 
peer-led and mission-driven—rather than 
prescriptive—nature of accreditation 
generally yields adequate evidence of student 
achievement from institutions and programs. 


Key takeaways on what accreditors reported 
with respect to the impact of accreditation 
standards and policies on the evidence of 
student achievement included: 


« Most accreditors encourage or require 
specific indicators of achievement 
such as graduation and completion 
rates; passing rates for certification, 
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licensure, or other standardized exams; 
and job placement rates. These are 
also the sources of evidence most 
cited by institutions and programs. A 
smaller share of accreditors prohibit 

or discourage particular sources of 
outcomes evidence, such as grades 
and indirect measures like student 
satisfaction. 

* Roughly half of accreditors (54 percent 
of programmatic, 46 percent of 
institutional) set bright-line quantitative 
threshold requirements for at least one 
student outcomes indicator. Twenty- 
three percent of programmatic and 
eight percent of institutional accreditors 
require members to set their own 
threshold requirements. 

* The majority of accreditors feel like 
their institutions and programs currently 
provide adequate evidence of student 
achievement for accreditation purposes 
and that the quality of evidence has 
improved in the last five years. 

« Most accreditors provide formal 
training along with a selection of 
other resources to assist members 
in examining student learning and 
achievement. 

« A large share of both programmatic 
and institutional resoondents reported 
that they have revoked or denied 
accreditation due at least in part to 
problems with evidence of student 
achievement. Additional research 
to clarify whether the gap is in 
actual achievement or in a lack of 
adequate data would be helpful so 
that appropriate supports could be 
developed. 


Key takeaways on how accreditors perceive 
the evidence of student achievement 
affecting the accreditation status of 
institutions and programs included: 


« Forty percent of programmatic and 
46 percent of institutional accreditors 
reported that more than a quarter 
of their actions in 2017 were in part 
related to problems with demonstrating 
evidence of student achievement. 

« Forty percent of programmatic and 
54 percent of institutional accreditors 


reported that programs and institutions 
lost or were denied initial accreditation 
status in the last five years due in part 
to problems with demostrating evidence 
of student achievement. 


The increased capacity, engagement and 
alignment of accreditors on the issue 

of student achievement should not be 
minimized. At the same time, there is room 
for further rigor in how accreditors work 
with institutions and programs to ensure 
meaningful information and insights about 
actual student learning and success. 


Key takeaways from accreditors on what is 
working, needs improvement and causes 
concern among accreditors about student 
achievement included: 


* Accreditors reported that training, 
coaching and workshops along 
with detailed guidance and clear 
expectations are working well. 
Institutional accreditors also 
emphasized their review processes 
and evaluation strategies, while 
programmatic accreditors emphasized 
flexibility in defining student outcomes 
and publishing student achievement 
metrics as working well. 

¢« Institutional and programmatic 
accreditors cited the need to 
improve the quality of measures 
used to demonstrate evidence of 
student learning and achievement— 
underscoring the challenge of balancing 
quantitative and qualitative measures, 
institutional/program diversity and 
recognition requirements from CHEA 
and USDE, and perceived tensions 
between “student learning” and 
“student achievement.” 

* Accreditors expressed near unanimous 
concern about the trend toward 
standardized “cookie cutter” definitions 
and approaches as well as the use of 
“blunt” measures without regard for 
the rich heterogeneity of the missions, 
cultures and student populations of 
institutions and programs. 


The results from this survey speak largely 
to the processes accreditors employ rather 
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than to whether these processes are 

yielding meaningful information and insights 
about actual student learning and success. 
Accreditors need to continue to determine 
and make public what constitutes robust 
evidence of student achievement in different 
settings and develop publicly shared means 
for identifying and improving—or sanctioning 
—substandard performance in relation to 
student achievement. 


Research and Future 
Steps 


With a commitment to advancing evidence 
and transparency about student learning 
outcomes, CHEA is positioned to work with 
constituents to further strengthen the role of 
accreditation in appropriately capturing and 
improving student learning outcomes as an 
integral component of educational quality. 
CHEA will do so through further research 
and convening that complements its work in 
the recognition of accrediting organizations, 
with the USDE and in advocacy that 
balances accountability and transparency 
with heterogeneity of mission and academic 
independence. 


More concretely, CHEA’s research efforts 
will align with key standards in its 2019 
Recognition Policy and Procedures that 

it holds accreditors to for recognition: 1) 
promotes academic quality and advances 
student achievement and 2) demonstrates 
public accountability for performance 

and transparency. CHEA’s research will be 
intentional and focused on supporting the 
ongoing areas of challenge and opportunity 
for accreditors, including demonstrating 
positive impact on student achievement. 


Below are examples of the types of questions 
CHEA may examine or encourage via 
roundtables, quantitative inquiries, interviews 
and case studies. 


Evidence of Student Learning 

Accreditors report improved clarity as well 
as rigor of standards and evidence provided 
by programs and institutions. What can 

we deduce about how these changes have 


affected actual student learning outcomes, 
comparing today’s student outcomes data to 
historical data? 


Acceptable Levels of Student Success 
There is concern among accreditors about 
over-standardization of measures of student 
learning outcomes, yet the public demands 
greater transparency. How can accreditors 
develop student learning expectations that 
identify acceptable levels of success in 
different circumstances—without setting 
one-size-fits-all bright lines—that can serve 
as a foundation upon which institutions and 
programs can layer mission-specific evidence 
of student achievement? 


Expeditious Action to Counter Substandard 
Performance 

Accreditors and peer review are uniquely 
situated to drive continual improvement 
across diverse institutions of higher 
education. What tools and practices can 

the community develop to foster greater 
transparency, clarity and timeliness around 
actions to assist institutions and programs 
with substandard student achievement? 


Timely Action to Improve Performance 

One underlying expectation of transparency 
is that it will promote efforts to improve 
student outcomes. With greater sharing, 
what models can be replicated for institutions 
and programs that are not deemed 
substandard, but are seeking to improve their 
performance? How can accreditors ensure 
any changes do not have unintended harmful 
impacts? 
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